Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for json functions #546

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

ritwizsinha
Copy link
Contributor

@ritwizsinha ritwizsinha commented Jan 17, 2025

Addresses #516

@ritwizsinha
Copy link
Contributor Author

ritwizsinha commented Jan 17, 2025

@JelteF I have added all the json functions present in duckdb to the sql file, is this fine or do we need to break this up. All of this is quite repeated and most of it is tests, so I doubt a big PR is a problem

@@ -17,3 +17,227 @@ CREATE AGGREGATE @[email protected]_count_distinct(anyelement)

CREATE DOMAIN pg_catalog.blob AS bytea;
COMMENT ON DOMAIN pg_catalog.blob IS 'The DuckDB BLOB alias for BYTEA';

-- json_exists
CREATE FUNCTION @[email protected]_exists(json VARCHAR , path VARCHAR)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems confusing that these functions expect a VARCHAR type for the json argument. I'd expect that argument to be of the json or jsonb (or have functions for both of them).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just added for json

"iceberg_snapshots", "delta_scan", "read_json", "approx_count_distinct"};
const char *function_names[] = {"read_parquet", "read_csv", "iceberg_scan", "iceberg_metadata",
"iceberg_snapshots", "delta_scan", "read_json", "approx_count_distinct",
"json_exists", "json_extract", "json_extract_string"};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems this list is missing a bunch of the functions that you added.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added

@@ -0,0 +1,144 @@
-- <JSON_EXISTS>
-- Test 1: Path exists in a simple JSON object
SELECT json_exists('{"a": {"b": 1}}', '$.a.b'); -- Expected: true
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think most of these tests simply test that DuckDB works correctly, not so much the pg_duckdb intgration.

I think for each function it should give us enough confidence into pg_duckdb to add one test with reasonable arguments, and then tests with the various combinations of NULL arguments. So while I really appreciate the effort to add extensive tests, I don't think they add much value and would in practice only increase test runtime.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reduced the tests

@ritwizsinha
Copy link
Contributor Author

@JelteF structures are not supported as types in pg_duckdb so not possible to implement functions like:

json_transform(json, structure)	Transform json according to the specified structure.
from_json(json, structure)	Alias for json_transform.
json_transform_strict(json, structure)	Same as json_transform, but throws an error when type casting fails.
from_json_strict(json, structure)	Alias for json_transform_strict.

Added all the other ones

@ritwizsinha ritwizsinha requested a review from JelteF January 22, 2025 14:14
@ritwizsinha
Copy link
Contributor Author

Curious why this fails on postgres 17

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants